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Abstract 

Background: This paper describes an innovative protocol for a type-ll hybrid effectiveness-implementation trial that 
is evaluating a smoking cessation telephone care coordination program for Veterans Health Administration (VA) 
mental-health clinic patients. As a hybrid trial, the protocol combines implementation science and clinical trial 
methods and outcomes that can inform future cessation studies and the implementation of tobacco cessation 
programs into routine care. The primary objectives of the trial are (1) to evaluate the process of adapting, 
implementing, and sustaining a smoking cessation telephone care coordination program in VA mental health 
clinics, (2) to determine the effectiveness of the program in promoting long-term abstinence from smoking among 
mental health patients, and (3) to compare the effectiveness of telephone counseling delivered by VA staff with 
that delivered by state quitlines. 

Methods/design: The care coordination program is being implemented at six VA facilities. VA mental health 
providers refer patients to the program via an electronic medical record consult. Program staff call referred 
patients to offer enrollment. All patients who enroll receive a self-help booklet, mailed smoking cessation 
medications, and proactive multi-call telephone counseling. Participants are randomized to receive this counseling 
from VA staff or their state's quitline. Four primary implementation strategies are being used to optimize program 
implementation and sustainability: blended facilitation, provider training, informatics support, and provider 
feedback. A three-phase formative evaluation is being conducted to identify barriers to, and facilitators for, 
program implementation and sustainability. A mixed-methods approach is being used to collect quantitative 
clinical effectiveness data (e.g., self-reported abstinence at six months) and both quantitative and qualitative 
implementation data (e.g., provider referral rates, coded interviews with providers). Summative data will be 
analyzed using the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) framework. 

Discussion: This paper describes the rationale and methods of a trial designed to simultaneously study the clinical 
effectiveness and implementation of a telephone smoking cessation program for smokers using VA mental health 
clinics. Such hybrid designs are an important methodological design that can shorten the time between the 
development of an intervention and its translation into routine clinical care. 
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Background 

Smoking is responsible for 435,000 deaths per year [1]. 
Persons with a mental health diagnosis have particularly 
high rates of tobacco use and consume over 46% of ciga- 
rettes sold in the US [2,3]. Patients with bipolar disor- 
der or schizophrenia have the highest smoking rates 
(69% and 58-90%, respectively) followed by those with 
post-traumatic stress disorder (45-63%) and depression 
(31-51%) [3-5]. Nicotine replacement therapy (NRT), bu- 
propion, and behavioral interventions are recognized as 
effective smoking interventions for patients with and 
without mental illness [6-10]. Although in-person behav- 
ioral counseling is recognized as the most effective form 
of behavioral intervention for smoking, telephone-based 
counseling for smoking cessation has greater reach than 
in-person counseling and is easily amenable to tailoring 
based on patient needs [9,11-13]. 

Every US state, as well as Washington DC, has a tele- 
phone "quitline," which residents can call to receive self- 
help materials, counseling, and, in some states, free or 
discounted medications. Unfortunately, telephone coun- 
seling has been underutilized among smokers with men- 
tal health diagnoses, and there are no published studies 
examining its effectiveness specifically in a mental health 
sample despite evidence that persons with a mental ill- 
ness accept, participate in, and benefit from telephone 
counseling programs for other purposes [14-16]. A sin- 
gle trial of telephone care coordination for smokers in 
primary care clinics that connected patients to their 
state quitline for counseling included patients with men- 
tal health diagnoses and showed that the program in- 
creased long-term abstinence compared with usual care 
[17]. However, the number of mental health patients in- 
cluded in the trial was too small to make conclusions 
about the programs effectiveness among mental health 
patients, and it is unknown whether the program would 
function similarly outside the primary care setting. In 
addition, it is unclear whether state quitlines can ad- 
equately serve smokers with mental illness, who may 
need more intensive treatment than the general popula- 
tion, especially those quitlines that offer only a single 
call or a small number of calls per client. A 2007 survey 
of state quitlines revealed that none of the US quitlines 
had specialized services for clients with mental illness, 
and only 6% had self-help materials tailored for smokers 
with mental health conditions [18]. Therefore, mental 
health patients may require a larger number of calls and 
more tailoring than is currently available from the ma- 
jority of state quitlines. 

We are conducting a hybrid effectiveness-implementation 
trial [19] to simultaneously study the clinical effective- 
ness and implementation of a telephone care coordin- 
ation program for smokers using Veterans Health 
Administration (VA) mental health clinics. Hybrid 



effectiveness-implementation study designs are an im- 
portant and innovative methodological advance de- 
signed to shorten the time between the development of 
an intervention and its translation into routine clinical 
care. According to Curran et al. [19], there are three pri- 
mary types of hybrid designs that lie on a continuum 
from clinical effectiveness research to implementation 
research. Type I designs focus on studying the clinical 
effectiveness of an intervention while observing and 
gathering information on the implementation of that 
intervention into routine care. Type II designs prioritize 
clinical and implementation outcomes equally by con- 
currently studying the clinical effectiveness of an inter- 
vention and one or more strategies for implementing 
that intervention. Type III designs focus on implemen- 
tation outcomes by studying one or more implementa- 
tion strategies for an intervention while gathering 
information about the interventions clinical outcomes. 
Our trial represents a Type II hybrid design. The trial 
will estimate the clinical effectiveness of the care co- 
ordination program with VA mental health patients, 
compare the effectiveness of quitline counseling to 
counseling from VA staff trained to work with smokers 
with mental illness, identify important barriers of and 
facilitators for implementing the program in VA mental 
health treatment settings, and describe key outcomes of 
four non-randomized implementation strategies. The 
trial has three primary objectives: (1) evaluate the pro- 
cess of adapting, implementing, and sustaining a smok- 
ing cessation telephone care coordination program in 
VA mental health clinics; (2) determine the effectiveness 
of the program in promoting long-term abstinence from 
smoking among VA mental health clinic patients; and 
(3) compare the effectiveness of telephone counseling 
delivered by VA staff trained to work with smokers who 
have a mental health condition with that delivered by 
state quitlines. This paper describes the design and 
methodology of the trial. 

Methods/design 

Overview 

Figure 1 provides an overview of the clinical effective- 
ness portion of the trials design. We are implement- 
ing a telephone care coordination program (called the 
Telephone Quality Improvement Trial in Mental Health, 
or "TeleQuitMH") at six VA facilities. Mental health pro- 
viders refer their smoking patients to the program via 
electronic medical record (EMR) consult. The program 
calls referred patients to offer enrollment using the pa- 
tients phone numbers listed in the EMR. Enrolled pa- 
tients receive smoking cessation medications and multi- 
session, proactive telephone counseling. In a one-to-one 
fashion, we randomize patients to receive the proactive 
telephone counseling from a VA counselor or their state 
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Figure 1 CONSORT flow diagram. 



quitline. We survey patients by phone at baseline, two 
months, and six months to assess outcomes. We are 
using four organizational- or provider-level interventions 
to optimize program implementation: blended facilita- 
tion, provider training, informatics support, and provider 
feedback. We are conducting a three-phase formative 
evaluation to identify barriers to and facilitators for pro- 
gram implementation. 

Study sites 

Table 1 describes the structure and target patient popu- 
lation of the six participating facilities in New York, 
New Jersey, Massachusetts, Vermont, New Hampshire, 
and Rhode Island. Each site includes one or more med- 
ical centers and multiple outpatient clinics. Like all VA 
sites, our participating facilities use a common EMR 
that includes clinical reminders for tobacco use screen- 
ing and treatment. Usual care for tobacco use at our 
sites includes patient access to NRT, Bupropion, and 
Varenicline through regular prescribing providers and 
in-person smoking cessation clinics offering group 
counseling. 

Participants 

All smokers who see a VA mental health provider at par- 
ticipating sites during our intervention period can be re- 
ferred to TeleQuitMH. The decision to refer a patient is 
based on the provider s assessment of the appropriate- 
ness of referral to a telephone program and the patients 
agreement to the referral. Eligibility criteria for study 
participation include referral from a VA mental health 
provider, access to a telephone, and access to a mailing 
address (a post-office box is acceptable). An estimated 
19,500 smokers use VA mental health services annually 
at our six sites. Based on a prior study testing a similar 
smoking care coordination program in VA primary care 
[17], we anticipate a 25% referral rate and a 50% enroll- 
ment rate among the referred, giving a total of about 
2400 patients enrolling in the program during our inter- 
vention period (1200 of whom will be randomized to VA 
counseling and 1200 to quitline counseling). 

Clinical intervention: telephone care-coordination 
program 

The TeleQuitMH care coordination program consists of 
the following sequential elements. The program begins 
with a provider-level intervention that enables the refer- 
ral of patients to tobacco cessation treatment. After the 
referral, patient-level treatment intervention begins. 

1. Simple EMR referral— The care coordination 
program takes advantage of VA usual care that 
requires providers to screen patients for tobacco use 
and either provide them with counseling or refer 
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Table 1 Overview of participating facilities 


Facility name 


States 
served 


Estimated number of MHC 
smokers 


% 
Female 


Number of affiliated medical 
centers 


Number of affiliated 
CBOCs 


R&D 
service 


New York Harbor 
HCS 


NY, NJ 


4,500 


8 


2 


6 


Y 


Bronx VAMC 


NY, NJ 


3,900 


6 


1 


4 


Y 


New Jersey HCS 


NJ 


5,000 


7 


2 


10 


Y 


Bedford VAMC 


MA 


2,300 


5 


1 


5 


Y 


White River Junction 
VAMC 


VT, NH 


1,800 


9 


1 


6 


Y 


Providence VAMC 


Rl 


2,000 


6 


1 


5 


Y 



HCS = Healthcare System; VAMC = VA Medical Center; CBOC = Community-Based Outpatient Clinic; MHC = Mental Health Clinic. 



them to treatment during routine visits. We are not 
attempting to standardize the screening process or 
counseling given by mental health providers. Our 
study introduces a new treatment referral option for 
providers— that of referring patients to TeleQuitMH 
via a new electronic consult in the EMR. Providers 
have the option to confirm the patients contact 
information and enter free-text information about 
their patient in the consult screen, but these 
responses are not required. Providers can access the 
consult either through the consult menu or through 
the tobacco use clinical reminder screens. 

2. Proactive outreach— Once our program receives a 
referral, we mail the patient a welcome packet that 
includes information about the TeleQuitMH 
program and study and a brochure containing all 
elements of informed consult. One week following 
this mailing, program staff call the patient using the 
phone number in his or her medical record. Staff 
make five attempts over two weeks to reach referred 
patients by phone to discuss the smoking cessation 
program, offer enrollment, and complete the verbal 
informed-consent process. Patients who enroll in 
treatment are provided with the next four elements 
of the program described below. Patients not 
interested in treatment have the opportunity to 
enroll in the follow-up surveys only. 

3. Medication coordination— All enrolled patients can 
receive mailed smoking cessation medications unless 
they decline or have contraindications noted by their 
referring provider. The method with which the 
TeleQuitMH program provides medications varies 
by site and is determined by a site Clinical Advisory 
Committee formed before site implementation. This 
committee consists of key stakeholders from the 
departments of Psychiatry, Social Work, Psychology, 
Pharmacy, and Smoking Cessation. There are three 
prescribing methods from which the Clinical 
Advisory Committee can choose: 

a. The site can designate one or two prescribers 
who will complete all prescriptions for 



participants at their site. Based on the success of 
the primary care-based projects that used a 
designated provider, we recommend this method 
to all sites. 

b. Referring prescribers prescribe for their patients 
at the time of referral to TeleQuitMH. If a patient 
was referred by a nonprescriber (e.g., a social 
worker), or a referring prescriber did not 
prescribe at the time of referral, a TeleQuitMH 
research assistant contacts the patients regular 
psychiatrist or primary care provider to request 
medications. 

c. A combination of methods 1 and 2 is used. 
Referring prescribers can prescribe for their 
patients at the time of referral. If a patient was 
referred by a nonprescriber or the referring 
prescriber did not prescribe at the time of 
referral, a TeleQuitMH assistant will contact a 
site-designated prescriber to request medications. 

4. Self-help materials— All enrolled patients receive a 
mailed self-help booklet developed jointly by the 
University of Colorado School of Medicine and 
Colorado Department of Public Health and 
Environment. The two-page booklet provides brief 
advice on preparing to quit smoking, setting a quit 
date, and preventing relapse. The booklet also 
provides corrective information on five common 
myths about smoking among persons with a mental 
health diagnosis. We tailored this booklet for a VA 
mental health population by including brief advice 
on linking smoking cessation treatment with regular 
mental health care and changing the booklets 
imagery to include pictures of persons in the 
military. 

5. Smoking cessation counseling— All enrolled patients 
have the opportunity to receive multi-call telephone 
counseling. To meet study Aim 3 described above, 
we randomize participants to receive this counseling 
from a VA counselor or from their state quitline. 
VA counseling: For patients randomized to receive 
counseling from a VA counselor, they are assigned a 
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VA counselor trained to work with smokers who 
have a mental health diagnosis. The counselor 
provides structured counseling using a protocol that 
we created specifically for this study. The protocol 
includes proactive counseling and relapse-sensitive 
scheduling. The content is based on motivational 
interviewing (MI) and problem solving therapy and 
addresses both behavioral and cognitive issues, 
including motivation, self-efficacy, difficult 
situations, comorbid mental health symptoms, 
coping strategies, medication usage, and relapse 
prevention. The counseling includes planning 
(pre-quit) and follow-up (post-quit) sessions. 
Planning sessions: The planning sessions last 
approximately 30-60 minutes and help patients 
develop an individualized quit plan. Content areas 
include smoking and quitting history, motivation, 
environmental factors, planning, proper use of 
quitting aids, and setting a quit date. Patients receive 
one to six planning sessions. 

Follow-up sessions: Patients receive follow-up calls at 
0, 1, 3, 7, 14, 21, and 30 days after their quit date. 
The calls are intended to prevent relapse and to help 
those who relapse or slip resume quitting. Follow-up 
calls, which last 10-15 minutes, also use a manual 
for consistency and fidelity monitoring. Participants 
are able to call the program staff in between 
scheduled calls if desired. 

State quitline counseling: For patients assigned to 
state quitline counseling, a research assistant 
initiates a "warm transfer" of the patient to their 
quitline via a three-way call to start the counseling 
process. After the initial three-way call, study 
personnel are not involved in any aspect of the 
quitline counseling. The quitlines follow their 
regular service protocols, which are described in 
Table 2. 

6. Follow-up— All patients enrolled in the study receive 
a call from a research assistant at two and six 



months. Patients who are abstinent from smoking 
are congratulated, but these assessment calls do not 
provide counseling. Patients who have relapsed at 
six months are asked if they want to go through the 
program again. 

Counseling standardization and fidelity 

State quitlines have standardized protocols and well- 
trained counselors providing counseling. For the VA 
counseling arm, the telephone counselors undergo an 
initial training consisting of two-to-four hours of MI 
training with a clinical psychologist and 20 hours of 
training on the study's clinical protocols. This training 
includes role-plays with each other and with the study's 
project director. After the project director has deter- 
mined that a counselor has met the role-play training 
objectives, the counselor advances to complete a series 
of standardized patient (SP) encounters. The SP encoun- 
ters involve calling an actor trained to portray a smoker 
enrolled in the study and completing planning and 
follow-up counseling sessions per study protocol. The 
SP encounters are audiotaped and reviewed by a clinical 
psychologist and the study's project director for adher- 
ence to protocols and counseling techniques. To ensure 
TeleQuitMH counseling standardization and fidelity 
after study implementation, the counselors complete 
clinical documentation using a template that allows 
them to indicate which protocol objectives were covered 
during each call and the timing of each call. The study's 
project director completes regular audits of the coun- 
selor call documentation to ensure adherence to the 
protocol. The counselors also attend weekly group 
supervision meetings with a clinical psychologist, during 
which time the counselors' active cases are discussed. 

Implementation strategies 

We are using a series of implementation strategies at 
each site to improve the implementation process and in- 
crease the uptake and sustainability of the TeleQuitMH 



Table 2 Overview of state quitline services 



State 


Typical number of 
counseling sessions 


QL-initiated 
calls 


Client- 
initiated calls 


Typical length of first 
session (min) 


Typical length of Follow-up 
sessions (min) 


Free 
medications 


MA 


6 


Y 


Y 


45 


20 


Patch 


NH 


4-6 


Y 


Y 


45 


20 


None 


NJ 


6 


Y 


Y 


30 


15 


None 


NY 


1 


Y 


Y 


20 


10 


Patch, gum 


Rl 


5 


Y 


Y 


45 


15 


None 


VT 


2 


Y 


Y 


30 


20 


Patch, gum, 
lozenge 


CT 


5 


Y 


Y 


15 


10 


Patch, gum, 
lozenge 



QL-initiated = quitline proactively calls clients to deliver counseling; client-Initiated = clients call into the quitline to receive counseling. 
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program. We used the Promoting Action on Research 
Implementation in Health Services (PARiHS) [20] and 
the Predisposing, Reinforcing, and Enabling Constructs 
in Educational Diagnosis and Evaluation (PRECEDE) 
[21] theoretical frameworks and evidence from the 
implementation science literature to guide our choice 
of implementation strategies. The PARiHS framework 
suggests that the success of implementing an evidence- 
based program into practice is determined by three 
factors: the evidence supporting the program, the imple- 
mentation context, and the process of implementation 
facilitation [20]. The PRECEDE framework suggests that 
changing provider behavior to use a new intervention is 
enhanced by strategies that predispose, enable, and 
reinforce behavior change [21]. Based on these frame- 
works, we chose four primary implementation strategies 
as described below: blended facilitation, provider train- 
ing, informatics support, and provider feedback. 

Strategy 7: blended facilitation 

Based on the PARiHS model, we use a combination of 
external and internal facilitation to implement the 
TeleQuitMH program. External facilitation is conducted 
by the study's principal investigator (PI) and project dir- 
ector, who meet with local stakeholders at each site to 
recommend methods for successfully implementing the 
care coordination program based on the methods found 
to be successful in the primary care-based projects. In- 
ternal facilitation is accomplished by each sites PI, who 
is responsible for serving as a local champion for the 
project or identifying someone at their site who would 
be appropriate to serve as a local champion. In addition, 
as described in the medication coordination section 
above, we form a Clinical Advisory Committee at each 
site to guide local implementation of the TeleQuitMH 
program. The committees include local representatives 
from the Departments of Psychology, Psychiatry, Social 
Work, Smoking Cessation, and Pharmacy. In addition to 
the committees' responsibilities in deciding the medica- 
tion coordination structure for their site, the committees 
provide guidance on how to inform local providers of 
TeleQuitMH, provide input on any foreseeable barriers 
to implementation and provider adoption, and decide 
how the TeleQuitMH program should work with or 
complement any existing smoking cessation programs. 
The committees meet regularly before site TeleQuitMH 
implementation and as-needed after implementation. 

Strategy 2: provider training 

Based on the PRECEDE model, to predispose providers 
to utilize the TeleQuitMH program, study Pis present 
the program to mental health providers at each site dur- 
ing regular staff meetings and via email newsletters. The 
presentations focus on presenting the evidence behind 



the program, including its effectiveness in VA primary 
care (including among mental health patients referred 
through primary care). The presentations also include a 
discussion on how the TeleQuitMH program functions 
within the context of each sites existing mental health 
structure and smoking cessation programs. 

Strategy 3: informatics support 

The PRECEDE model posits that enabling interventions 
simplify the implementation of a recommended practice 
for providers. To enable the use of TeleQuitMH, we cre- 
ate a local electronic consult for the program that allows 
providers to refer patients with three-to-four extra clicks 
as they complete a regular clinical progress note in the 
EMR system. The consult is also attached to the tobacco 
clinical reminder and mental health assessment tem- 
plates in the EMR system, with which providers are 
already familiar. For providers who do not have clinical 
privileges to send consults, the study's project director 
also accepts referrals via secure email. 

Strategy 4: provider feedback 

To reinforce use of the TeleQuitMH program, we use 
two strategies. First, we document all patient outreach 
and counseling sessions with a clinical progress note that 
mental health providers can read in the EMR system. 
We also add referring providers as "additional signers" 
to key TeleQuitMH progress notes, such as notes 
documenting their patients' enrollment in the program, 
their patients' quit plans, and their patients' completion 
of counseling. This allows providers to see their patients' 
progress toward quitting smoking and to view the 
TeleQuitMH program as a regular source of care at their 
VA facility. Second, we send a regular TeleQuitMH elec- 
tronic newsletter from a local opinion leader to all men- 
tal health providers at each site. The newsletters include 
instructions for referral and detail site-specific progress 
with the intervention. 

Patient measures 

Table 3 provides details on planned patient measures 
and timing of administration. Participants complete a 
survey at baseline, two months, and six months after 
enrollment. Participants complete the surveys over the 
telephone with research assistants. We make 10 at- 
tempts over one month to reach participants to 
complete the surveys. The surveys include a demograph- 
ics questionnaire designed by the study team; a smoking 
behavior and history questionnaire designed by the study 
team; the BASIS -24 scale measuring recent mental 
health functioning [22]; an impulsivity scale [23]; a func- 
tional health literacy scale [24]; measures of readiness 
to quit, motivation, self-efficacy, and attitudes about 
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Table 3 Patient measures and assessment schedule 



Measures 


BL 


2m 


6m 


Sociodemographics 


X 






Current and historical smoking behavior 


X 


X 


X 


Readiness to quit 


X 


X 


X 


Quitting self-efficacy and motivation 


X 


X 


X 


Medication assessment: Current use, contraindications to NRT or Bupropion 


X 






Use of prior smoking cessation treatment 


X 


X 


X 


Nicotine dependence 


X 


X 


X 


Alcohol and substance abuse 


X 


X 


X 


Smoking environment: Social support for quitting, household smokers, household smoking rules, employer smoker rules 


X 






Attitudes about smoking 


X 






Mental health symptoms (BASIS-24) 


X 


X 


X 


Health literacy 


X 






Impulsivity 


X 






Patient satisfaction with the program 




X 


X 


Patient assessment of treatment fidelity/counseling content 




X 


X 



BL = baseline; 2m = two months after enrollment; 6m = six months after enrollment. 



smoking developed by the study team; and satisfaction 
with treatment received from the TeleQuitMH program. 

Formative/implementation evaluation 

The study includes a formative evaluation to help us 
modify the care coordination program and implementa- 
tion strategies during our study period, to enhance the 
interpretation of summative data, and to better inform 
future implementation efforts. Table 4 provides details 
on formative evaluation activities and measures. We 
used the Stetler et al. [25] model to guide the develop- 
ment of our formative evaluation methods. This model 
identifies four stages of formative evaluation for imple- 
mentation research: 

1. Developmental phase— -Based on our experience 
with the primary care-based projects, we anticipate 
that it will take six months to implement the 
TeleQuitMH program at a site. This time includes 
identifying a local champion, obtaining Institutional 
Review Board approval, implementing the 
TeleQuitMH programs main components, and 
informing providers. During this time we will 
conduct mental health provider surveys and 
interviews with site stakeholders to measure 1) the 
level of organizational readiness of the participating 
clinics both in general and specific to the 
intervention, 2) perceptions of the evidence 
supporting smoking cessation telephone counseling 
and smoking cessation medications, 3) perceptions 
regarding the feasibility and utility of the planned 
intervention, and 4) perceptions of anticipated 



barriers or facilitators to the successful 
implementation of the intervention. We will also 
assess the extent to which each sites local champion 
and Clinical Advisory Committee are interested in, 
and actually perform, their project responsibilities, 
assess the structure and processes of the sites state 
quitline operations (e.g., hours of operation), and 
assess the type and intensity of local smoking 
cessation services already available to mental health 
patients. We will formulate modifications to the 
TeleQuitMH program structure and our provider 
outreach/marketing efforts to address findings from 
these activities. 
2. Implementation/progress-focused phase— -For the 
purpose of our evaluation, we will combine Stetler 
et al.s implementation and progress-focused phases. 
After implementing the TeleQuitMH program at a 
site, we assess several patient and provider variables 
on a regular basis, including the number of patient 
referrals, the percentage and types of providers who 
are referring patients, the percent of patients 
enrolling and engaging in treatment, and patient 
satisfaction with the program. We also conduct a 
second provider survey during this phase evaluating 
reasons for a provider s referring or not referring 
patients to the program, provider satisfaction with 
various aspects of the program, and what, if any, 
recommendations providers have for changes to the 
program that they feel would improve its 
effectiveness. We also assess referral trends in 
relation to our provider outreach activities in order 
to modify or target future outreach efforts. Last, we 
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Table 4 Formative evaluation (FE) measures and evaluation schedule 



FE Measure/ 
activity 



Measure/activity description 



Impl./ Interpret./ 
Develop- progress sustainability 
mental focused focused 



Reach/Implementation Analysis 



Referral rates 

Treatment 
uptake 

Exposure to 
TeleQuitMH 



Number of patients referred on weekly and monthly basis. 

Percent of referred patients enrolling in treatment, engaging in treatment, and 
completing treatment. 

Percent of MH patients for whom tobacco use screening was completed. 



Patient Process Data 

Enrollment Rates Rates of enrollment to TeleQuit MH and State quitlines 

TeleQuitMH Self-reported satisfaction with TeleQuitMH structure and treatment (including VA 

Satisfaction versus quitline satisfaction) ; assessed 2m and 6m after enrolling 

Provider Process Data 



Provider Survey 



Stakeholder 
Interviews 



Perceptions of program marketing; how likely they will be to use the program; 
barriers to implementation; suggestions for improvement; provider use of the program; 
program satisfaction; perceived effectiveness of activities to facilitate implementation; 
impact of the program on their patients/client. 

Perceptions of program marketing; how likely they will be to use the program; barriers 
to implementation; suggestions for improvement; provider use of the program; 
likes/dislikes about the program; impact of the program on their patients/clients. 



Referral rates/ 
patterns 

MH Unit Data on Implementation Context 



Percent of providers referring; Number of referrals from each provider; Rates by 
provider type 



VA Survey of MH 
Programs[28] 

MH Unit 

Performance Data 
[29] 

Implementation Facilitation - Structure 

Site Champion 



Clinic's structure, size, staffing, environment, and resources for meeting MH and 
tobacco cessation treatment goals. 

Clinic-level performance on VA MH and tobacco performance measures 



Site Clinical 



Presence of a site champion, level of interest in performing site champion duties, 
position at the VA, role on the project 

Presence of a site Clinical Advisory Committee, level of interest in performing 
Advisory Committee Committee duties, Committee members positions at the VA & roles on the project 

Site smoking 
cessation services 

TeleQuitMH 
Structure 

State quitline 
Structure/Issues 



IRB Issues 



Presence, types, and structure of site smoking cessation clinic and other local smoking 
cessation services 

Structure of the TeleQuitMH program at each site, including design and appearance of 
the tobacco cessation clinical reminder, prescribing structure, and referral mechanisms 
(e.g., consult design, email referral option) 

Structure of quitline services; issues/problems identified in connecting TeleQuitMH 
patients to quitline services. 

Time from first submission to receiving IRB approval; type of submission required (full 
board review, expedited, exempt); site-specific requirements that may affect TeleQuitMH 
implementation and adoption (e.g., detailed consent process) 



Implementation Facilitation - Activities 

Marketing Efforts Provider outreach efforts, academic detailing, provider feedback. 



Treatment Fidelity 



Extent to which the TeleQuitMH staff are providing the telephone counseling, 
medication coordination and warm-transfer to the state quitlines as intended. 



X 
X 



X 
X 
X 



X 
X 



MH = mental health. 
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continue to assess the impact of each site's 
TeleQuitMH components (e.g., prescribing 
structure, referral process) on implementation 
fidelity, referral rates and patient uptake, treatment 
fidelity by the VA counselors, and the presence and 
quality of facilitation provided by the local 
champions and Clinical Advisory Committees. 
3. Interpretation/ sustainability ^-focused phase— 
Following the intervention period at each site, we 
collect two additional measures: an organizational 
survey of mental health programs, which assesses 
each sites mental Health clinic structure, size, 
staffing, environment, and resources [26]; and 
mental health unit performance rates on the mental 
health treatment and tobacco use clinical reminders 
required by the VA [27]. We use these data and the 
formative evaluation data collected during the 
previous two phases to describe the local 
implementation context, to formulate interpretations 
of primary outcomes data, and to create 
recommendations for future implementation efforts 
of the telephone care coordination program. 

Summative analyses 

We are using the RE-AIM framework to guide summa- 
tive data analyses [28]. The RE- AIM framework suggests 
that it is crucial for a system-level intervention to con- 
sider the following: Reach— the proportion of patients 
reached by the intervention; Effectiveness— -the effective- 
ness of the program at meeting clinical goals; Adoption 
—the proportion of providers and patients that used the 
intervention; Implementation— -the extent the interven- 
tion was implemented; and Maintenance— whether the 
intervention was maintained over time. 

Reach 

To assess the reach of the intervention, we will deter- 
mine the proportion of mental health patients who were 
exposed to the TeleQuitMH program. To do this, we 
will first use the EMR system to identify all smokers at 
participating facilities who had a mental health visit dur- 
ing the intervention period. This group — all smokers 
with a mental health visit — will be our denominator. If 
the TeleQuitMH-adapted tobacco use clinical reminder 
appeared for the patients mental health care provider, 
we will consider that patient to have been exposed to 
TeleQuitMH. Reach will be calculated as the number of 
smokers exposed to TeleQuitMH divided by the total 
number of smokers with a mental health clinic visit. 

Effectiveness 

Study Aims 2 and 3 examine the clinical outcomes of 
the TeleQuitMH program. For Aim 2 (Determine the ef- 
fectiveness of telephone counseling in promoting long- 



term abstinence from smoking among mental health pa- 
tients) there will be no control group, as these data will 
only be collected on patients who enroll in TeleQuitMH. 
We will calculate the percentage of participants with 
30-day point prevalence abstinence at two and six 
months. We will also examine which baseline character- 
istics (e.g. age, race/ethnicity, self- efficacy, nicotine ad- 
diction, mental health diagnosis, symptom severity) are 
associated with abstinence, and we will use multivariable 
logistic regression to determine those characteristics that 
are independently associated with abstinence. For Aim 3 
{Compare the effectiveness of telephone counseling deliv- 
ered by VA staff with that delivered by state quitlines) 
we will calculate the 30-day point prevalence abstinence 
rates at two and six months separately for the two treat- 
ment arms and compare these rates using logistic regres- 
sion, adjusting for covariables. 

We based our power calculation on an ability to detect 
a difference in six-month abstinence between our two 
randomized groups. We expect the six-month abstin- 
ence rate will be 15% in the quitline arm and 20% in the 
VA arm. With an anticipated 1200 participants in each 
treatment arm, we will have >80% power to detect this 
5% difference in abstinence rates between the two arms 
at the 0.05 significance level. 

Adoption 

We will calculate program adoption by providers and 
patients across all sites and for each site separately. For 
this study, adoption by a provider will be defined as 
referring one or more patients to the TeleQuitMH 
program; i.e., the provider used our provider-level inter- 
vention of adapting the EMR to enable referrals. To cal- 
culate the program adoption by providers, we will 
generate a list of all mental health providers at each site 
using administrative records and a list of all mental 
health providers who made at least one referral using 
study data. We will then divide the number of providers 
who made at least one referral by the total number of 
providers. Adoption by a patient will be defined as enrol- 
ling in treatment. We will have a list of all patients re- 
ferred to TeleQuitMH and the list of patients who enroll 
in treatment. We will divide the number of patients who 
enroll in treatment by the total number of patients re- 
ferred to the program. 

Implementation and maintenance 

We will assess the program implementation at each site 
by examining the activities occurring at each site. There 
are eight main implementation activities that need to be 
completed to fully implement the TeleQuitMH program 
at a site (Table 5). We will score each site on the com- 
pletion, fidelity, and intensity of each of these eight ac- 
tivities using anchors developed by the study team. We 
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Table 5 Major TeleQuitMH implementation activities at 
each site 

1 . Designation of a local champion for the program 

2. Creation of a site Clinical Advisory Committee 

3. Creation of site prescribing structure 

4. Modification of EMR system to enable referrals and local 
documentation of treatment delivery 

5. Permission and process to make warm-transfers of patients to the 
state quitline 

6. Program marketing to providers 

7. Provider referrals 

8. Delivery of smoking cessation counseling and smoking cessation 
medications to enrolled patients. 



will assess program maintenance by examining monthly 
referral rates to examine time trends in referrals and 
whether patients and providers continue to use the pro- 
gram for the full intervention period. 

Formative evaluation analyses 
Developmental phase 

Baseline provider survey and stakeholder interview data 
on attitudes toward smoking cessation and perceived 
barriers to telephone smoking cessation will be used to 
refine the implementation of the intervention. Docu- 
mentation of the core structural components of the im- 
plementation at each site (e.g., prescribing structure, 
pre-existing smoking cessation services, characteristics 
of local champion) will be used to ensure the best pos- 
sible implementation given site circumstances. 

Implementation/progress focuses 

Referral rates, treatment uptake and exposure at the 
patient level, and responses to TeleQuitMH at the pro- 
vider level (collected via surveys, stakeholder inter- 
views, and referral rates/patterns), will be continuously 
monitored to track program progress and identify sites 
that may need additional activities and efforts focused 
on facilitating implementation. Information on the 
structure of the implementation as it unfolds will be 
reviewed at regular staff meetings and linked with ini- 
tial patient (enrollment rates, satisfaction) and provider 
process data (referral rates and patterns) to determine 
next steps for implementation including directed out- 
reach/marketing; changes in structure, etc. 

In terpretation/sustainability phase 

After the intervention period has ended at each site, we 
will use the interpretation/sustainability phase of our 
formative evaluation to interpret primary outcomes, 
using qualitative pattern-matching approaches to explore 
how the implementation activities and structure of the 
implementation (including treatment fidelity) may help 



explain variations in outcomes at the site level. Simi- 
larly, data on the implementation context will be ana- 
lyzed to determine variation across sites and whether 
that variation is associated with both the nature of the 
implementation and the outcomes achieved. Formative 
evaluation data will be analyzed longitudinally to deter- 
mine if changes in implementation are associated with 
changes in process variables (e.g., reach, uptake). In 
terms of sustainability, analyses will focus on identifying 
the site and implementation features and characteristics 
that facilitated and hampered effectiveness. Finally, de- 
scriptions of the degree to which core elements of 
TeleQuitMH were integrated into systems of care at 
each site will be used to characterize the sustainability 
of TeleQuitMH and to identify the local conditions 
which facilitate replication and dissemination. 

Economic evaluation 

It is not within the scope of this project to conduct a 
cost-effectiveness analysis. For this project, we will assess 
the costs involved with the TeleQuitMH program, in- 
cluding staff costs (salary and fringe benefits), cost of 
smoking cessation medications, and VA overhead. We 
will use data from the VAs Decision Support System for 
the costs of provider time and smoking cessation medi- 
cations. We will calculate the cost associated with coun- 
seling delivery using study administrative records for the 
VA counseling arm and quitline program estimates for 
the quitline arm. We will use VA rental space agree- 
ments at our primary site to estimate the cost of VA 
overhead associated with the program. 

Discussion 

It can take 17 years for new clinical interventions to 
make their way through the efficacy-effectiveness-imple- 
mentation research pipeline and become usual clinical 
care that improves patient outcomes in routine practice 
settings [29]. Hybrid effectiveness-implementation study 
designs offer the potential to greatly shorten this transla- 
tion time [19]. This study utilizes an innovative hybrid 
design to concurrently study the clinical outcomes and 
implementation into routine care of a smoking cessation 
telephone care coordination program for mental health 
patients in six VA facilities. The study will study the clin- 
ical effectiveness of the care coordination program and 
compare the effectivness of two models for connecting 
mental health patients to telephone smoking cessation 
counseling (a tailored VA-staffed program versus a 
warm-transfer of patients to their state quitline), while 
producing important data about the acceptability and 
feasibility of telephone-based smoking cessation treat- 
ment among mental health providers and patients. For- 
mative evaluation data collected during study start-up 
and regularly throughout the intervention period will 
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help future tobacco control efforts in mental health 
treatment settings by identifying barriers to and facilita- 
tors of successful treatment program implementation. 

The study will have several limitations. First, we will 
rely on self-reported abstience, which may produce over- 
estimates of short- and long-term quit rates among par- 
ticipants. Second, there is considerable variability in 
services provided by the quitlines, and we will not have 
sufficient power to conduct Aim 3 analyses separately 
for each state. Third, we randomize at the patient-level 
and will not be able to determine the effectiveness of 
our implementation strategies. However, researchers can 
use the qualitative and nonrandomized implementation 
outcomes data collected during the trial to design future 
group-randomized studies evaluating the effectiveness of 
different stratagies for implementing tobacco cessation 
treatment programs in mental health clinics. Lastly, we 
will not have data on the number or duration of coun- 
seling sessions provided to participants connected to the 
quitlines for counseling, so we will be unable to account 
for treatment intensity in our Aim 3 analyses. 
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